A 3D display device includes, as building elements, a lenticular element (a lenticular lens element), and a polarized optical element for parallax stereogram, integral photo, and the like, and enables observation of a 3D image. The 3D display device displays an image corresponding to each viewpoint by a screen-divisional scheme or time-divisional scheme, and guides light from each image to each viewpoint.
There is proposed a service of distributing image data (image contents) for 3D display via a communication network such as a digital broadcasting network or the Internet. In distributing an image, depth data of the 3D space and data containing a large number of multi-viewpoint image groups are distributed. From the distributed data, the user can generate a composite image (to be described later) complying with a user's 3D display device by an arbitrary viewpoint image generation process (to be described later). Image data can, therefore, be distributed regardless of the amount of parallax of a 3D image which changes depending on a pixel arrangement style defined by a 3D display device, the number of viewpoints, and other parameters and specifications (to be referred to as 3D display parameters collectively hereinafter).
The arbitrary viewpoint image generation process is a process of inputting two or more viewpoint images, and generating an image corresponding to a viewpoint different from the input viewpoint images. One method of the arbitrary viewpoint image generation process use a 3D model. To the contrary, an arbitrary viewpoint image generation process which does not use any 3D model is also called an image-based rendering process, and has enthusiastically been studied these days. As a typical example of this process proposed by Seitz et al., a warping process is done using depth data of the 3D space that is generated from a correspondence extracted from two viewpoint images, and an intermediate viewpoint image is generated (Steven M. Seitz, C. R. Dyer, View Morphing, Proc. SIGGRAPH 96, pp. 21-30, 1996).
Considering efficient use of a communication network, it is very redundant and inefficiency to directly distribute depth data of the 3D space of each scene and a multi-viewpoint image group. Thus, for a screen-divisional type 3D display device, it is more practical to generate a composite image (3D image) so as to comply with a specific 3D display device, and distribute the composite image.
A composite image is final image data for presenting a 3D image by using a 3D display device. An example of the composite image is shown in FIG. 3 (details thereof will be explained in the following embodiment). FIG. 3 illustrates an example of a composite image at five viewpoints.
When each pixel of a display is made up of three, red (R), green (G), and blue (B) subpixels, as shown in FIG. 3, adjacent viewpoint images of different R, G, and B subpixels are displayed to prevent color misregistration on the observation plane (plane on which viewpoints are aligned) (see Japanese Patent Laid-Open No. 57-11592).
Further, when lenticular lenses are arranged so that the generatrix directions of lens parts match the vertical direction, or the apertures of a parallax barrier are aligned in the vertical direction, the vertical resolution of a presented 3D image decreases. The decrease in the resolution of a 3D image is suppressed by arranging lenticular lenses so that the generatrix directions of lens parts extend obliquely, or properly distributing barrier apertures in the vertical and horizontal directions (see U.S. Pat. No. 6,064,424).
As described above, the pixel arrangement required for a composite image changes depending on the number of viewpoints of a 3D display device, and also depending on 3D display parameters such as the configuration and display scheme of the 3D display device even when the number of viewpoints is kept unchanged.
However, there has not conventionally been proposed any method of observing a composite image generated for a specific 3D display device on another 3D display device having different 3D display parameters. Image contents which are generated and distributed for a specific 3D display device can be observed on only the specific 3D display device. In this situation, the portability of image contents between 3D display devices having different 3D display parameters is very poor.